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Abstract. We argue that one ot the early goals of color vision is to distinguish one kind 
of material from another. Accordingly, we show that when a pair of image regions is such 
that one region has greater intensity at one wavelength than at another wavelength, and the 
second region has the opposite property, then the two regions are likely to have arisen from 
distinct materials in the scene. We call this material change circumstance the "opposite 
slope sign condition." With this criterion as a foundation, we construct a reoresentation of 
spectral information that facilitates the recognition of material changes. 

Our theory has implications for both psychology and neurophysiology. In particular, Heriny's 
notion of opponent colors and psychologically unique primaries, and Land's results in 
two-color projection can be interpreted as different aspects of the visual system's goal of 
categorizing materials. Also, the theory provides Wjo basic interpretations of the function of 
double-opponent color cells described by neurophysiologists. 
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1. Introduction 

The human visual system performs a remarkable feat. The pattern of light that reaches 
the eye from a scene is the result of a complex interaction among several factors: the quality 
of the illuminant, the geometry of the scene, and the properties of the materials composing 
the visible surfaces. Yet somehow these confounded factors are mostly separated in our 
perception. We see particular spatial arrangements of objects. These objects appear 
bounded by surfaces having properties— color and texture— roughly invariant over a range 
of conditions of geometry and illumination. To compute invariant descriptions of the material 
properties of surfaces is an important goal of any visual system. Such material descriptors 
are useful for object recognition and visual search. 

It's commonplace to assume color vision has something to do w/ith capturing the 
albedoes of surface materials.^ But exactly what aspect of the albedo function would serve 
a visual system best? Consider the grandiose goal of recovering a material's albedo as a 
continuous function of wavelength. Not only is this goal impractical; it is counter to the 
aim of finding invariant descriptors. With such an over-zealous representation, unimportant 
variations in a surface would prevent its being recognized as a single region, a patch of 
one kind of stuff. The perception of the world would be shattered with spectral acuity too 
fine; one literally wouldn't be able to see the forest for the trees. 

Here we seek a representation of material reflectance in which trivial surface variations 
can be overlooked in order to appreciate important similarities.^ At the same time, the 
representation must allow some discrimination among different materials. Below we develop 
such a categorical color space, based on a theoretical solution to the problem of identifying 
material changes. A trichromatic system, it will be shown, yields a two-dimensional color 
space in which the axes will turn out to represent boundaries between different materials. 
The four quadrants of the two-dimensional space represent material categories. 



2. Spectral Information at Edges 

When two image regions arise from different materials in the scene, the transition from 
one material to another will usually bring about an edge in the image. Thus we restrict our 
search for material changes to edges. How can we decide whether an edge is due to a 
material change? 

An edge in the image will usually arise from a single event or state of affairs in 
the three-dimen sional scene (Marr, 1882). The most common edge types are shadows, 

^The albedo of a material is a function of wavelength ^(X), with range (0,1), that indicates what 
fraction of photons (emitted by some light source) at each wavelength will be reflected. 

^We are not suggesting any spectral information be thrown away. We are merely exploring a single 
problem. Other problems may require detailed spectral information. 
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Figure 1 Graphs of image intensity versus wavelength. Each curve represents the image intensity 
measurable from one image region. A) Two graphs of same shape: a likely lawful change. B) Two 
graphs of different shape: a candidate for material change. 



highlights, surface orientation discontinuities, and pigment density changes."' Alternatively, 
an edge may be due to a material change, a discontinuity between two different kinds of 
stuff.'* How can a material change edge be distinguished from other types of edges? Rubin 
& Richards (1982) attempted to answer this question. Edges which arise from shadows, 
orientation changes and highlights are lawful in the sense that there are equations that 
describe how image intensities will change across these edges. By contrast, material 
changes are completely unpredictable; they are arbitrary changes, and as such, can only 
be inferred by ruling out, at a given edge, the possibility of any of the above lawful changes. 

To infer material changes, we now face the awkward prospect pf having to reject, 
one by one, each of the lawful changes. Perhaps there is some method of rejecting all of 
those edges en masse. Fortunately, there is a simple ordinal rule common to all the edges 
formed by lawful processes: if the intensity at one wavelength decreases across a lawful 
edge (shadows, highlights, and so on) then the intensity must also decrease at all other 
wavelengths taken across the same edge (Rubin and Richards, 1982). When this condition 
s violated, we say there is a "spectral crosspoint" across the edge. Spectral crosspoints 
mply material changes; a spectral crosspoint is illustrated in Fig. 2a. The spectral crosspoint 
s not the only means of discovering material changes, however. We will show that a second 
and independent condition holds for each of the lawful processes— namely the preservation 
of ordinality of image intensity across wavelength. A violation of this condition implies a 
material change. 



•^Surface orientation change and shadow can coincide at an edge, but this exception is unimportant 
to the arguments that follow. See F^ubin & Richards, 1932, footnote 16. 

■'We consider materials to consist of some spectrally neutral embedding material (e.g., cellulose) 
impregnated with a single pigment (e.g., chlorophyll). A material change is a change in pigment type, 
or a change in both pigment and embedding material. 



RUBIN AND RICHARDS COLOR AND MATERIAL CATEGORIES 

3. The Opposite Slope Sign Inference 

3. 1 The Lawful Processes 

Figure 1a shows two image intensity graphs of the same shape. Intuitively, the two 
graphs, of similar shape, arise from measurements taken on either side of a "lawful" edge 
type. Figure lb shows two graphs of different shape. None of the lawful edge types could 
have produced such a distortion, and intuitively it seems that a material change edge Is 
the best explanation. We now must make explicit what we mean by "same shape" and 
then show that this definition of spectral shape remains invariant across edges created by 
shadows, changes in surface orientation, highlights or variations in pigment density— namely 
the lawful conditions we wish to reject as material changes. 

Definition: Two curves of intensity versus wavelength have the same shape if the 
ordinal relations of image intensity across wavelength are preserved. 

Thus, if /x(X) and /r(X) are image intensities as functions of wavelength measured on both 
sides, X and Y, of an edge. /^(X) and /r(X) have identical ordinality if, for all Xi and X2, 
^x(Xi) < /x(X2) iff /y(Xi) < /y(X2). Note that two image intensity functions of identical 
ordinality will have local extrema at the same values of wavelength. 

Given this ordinal definition of "same shape", Appendix 1 shows that the ordinality 
relationship is preserved across all edges arising from the lawful edge types, provided that 
the following two conditions hold: 

Gray world condition: The average of all the different albedoes in the scene will 
be a spectrally flat "gray", so that the diffuse reflected light will have the same 
spectral character as the direct light. 

Spectral normalization: The spectral samples of image intensity have been 
normalized with respect to the color of the illuminant. 

(The need for the second condition, namely spectral normalization, will be eliminated 
subsequently.) 



3.2 The Opposite Slope Sign Operator 

We now can proceed to test for "same shape" using the ordinality relation. If ordinality 
is violated across an edge, then we infer the edge does not arise from one of the "lawful" 
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Figure 2 Graphs of image intensity (ordinate) versus wavelength (abscissa). Two wavelength 
samples Xi and X,, are shown. An image region yields two samples of intensity, one for each 
wavelength, ana is represented by the line segment connecting the two sample values a) & c) Two 
examples of the spectral crosspoint (Rubin & Richards, 1982). a) & b) Two examples of the opposite 
slope sign condition. This is the minimal configuration that shows different ordinalities Note tfiat 
he crosspoint and opposite slope sign condition are completely independent, since they can occur 
together (a), or each can occur alone (b and c), or neither can occur (d) 



processes and hence must represent a material change (provided also, of course, that our 
grey world condition is not violated).^ 

What is the simplest way to seek violations of ordinality? A pair of spectral samples 
suffices. Let the image intensities on both sides of an edge be measured at wavelengths 
Xi and X2. If image intensity at X, is greater than that at X2 on one side of the edge, then 
the ordinality condition requires the same relationship hold on the other side. So if the 
two sides of the edge do not have greater intensity in the same spectral sample, ordinality 
is violated; the edge cannot be lawful. (Details are given in Appendix 1.) This condition 



Mt IS possible when the grey world assumption is wrong, material changes will be inferred from 
images. This is not entirely bad news; if human perception also goes awry when the grey world 
assumption is violated, then our theory will become more credible as an account of biological visual 
systems. 
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is called the opposite slope sign condition.^ Examples are shown in Fig. 2a and 2b. The 
"slope" Of the opposite slope sign condition is the slope of the graph of intensity versus 
wavelength; it is an evaluation of the sign of the derivative of the spectral image intensity 
function, ^. 

More formally, given two regions X and Y across an edge and intensity samples / 
taken at two wavelengths X, and X2, we have the following test for a material change: 

Opposite Slope Sign Condition: 

{Ix\, - Ix\,)(Jy\, - /rxj < 0. 

which may be contrasted with' the previously derived crosspoint condition (Rubin and 
Richards, 1982): 

Spectral Crosspoint Condition: 

{Ix\, - /rxi) (/xx3 - Jy\,) < 0. 

iNoit: mm uie opcoiicii ui ubiipOini ciiia liie upposiie siupe jsiyn (junumoiis are cumpieiery 
independent. Figure 2a shows the tv/o occurring together. Each condition can arise alone, 
as shown in Figs. 2b and 2c. Finally neither condition is necessary, as shown in Fig. 2d. 

The two conditions are related by a kind of symmetry. The spectral crosspoint must 
make two comparisons across an edge (one for each wavelength), and combine them 
logically (both comparisons must work out in the correct way). The opposite slope sign 
condition must make two comparisons, one within each image region, and then combine 
them logically across the edge. 

To summarize: the spectral crosspoint~our original means of finding material changes- 
has been augmented by a second and independent material change condition: opposite 
slope sign. The opposite slope sign condition is the key theoretical result on which Vv^e will 
base our spectral representation of material types. We choose opposite slope sign rather 
than the crosspoint, because the opposite slope sign condition tells us something about 
each of the two regions that produce it. Namely, one region has positive spectral slope, the 
other negative. By contrast, the spectral crosspoint cannot be decomposed into assertions 
about t he two regions that produce it. In a crosspoint, spatial and spectral information cire 

''The opposite slope sign condition is described here as existing statically, across an edge. It is a 
spatial comparison of spectral infcrrr-ation. A comparison of spectral information in time is equivalent. 
Such a temporal opposite slope sign condition would work as follows: An eye could sweep across 
an edge, and the spectral information before and after the movement couid be compared. Similarly 
there is a temporal equivalent of the crosspoint. Consequences of these isomorphic computations in 
the temporal domain will not be explored here. 
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hopelessly intertwined. We do not cast aside the crosspoint, though. It will play a vital role 
in correcting for the spectral content of the illuminant. 



4.0 Spectral Normalization 

For the opposite slope sign test to find material edges successfully, it is necessary 
for the measured spectral intensities to be normalized. That is, these samples must be 
transformed to what they would have been under a spectrally flat ("white") illuminant. 
Clearly if no correction is applied, then the stronger spectral skew of an illuminant may 
not only reduce the number of observed opposite slope sign pairs, but more seriously, may 
transform pairs having the same slope sign into pairs that are seen as having an opposite 
slope sign. 

By contrast, the spectral crosspoint condition is insensitive to the spectral content 
of the illuminant, as can be seen by inspecting panels A and C of Fig. 2. (See Rubin 
& Richards, 1982, for a more formal treatment.) We capitalize on this property of the 
crosspoint to devise a theory of spectral normalization. Once the image has been spectrally 
normalized, it is as if the illuminant were white. The opposite slope sign condition will now 
be able to find correctly a maximum number of material changes. 

Consider now a scene composed of a large number of randomly selected materials. For 
each image region (simple closed curves defined by edges), take two samples of intensity 
/xi and /x^ at wavelengths Xi and Xg. Each region will be associated with a spectral slope 
sign, which is just the sign of the difference h, - h^. If the illuminant were white (same 
photon flux at all wavelengths), we would expect to have roughly equal numbers of regions 
of positive spectral slope and regions of negative spectral slope. This expectation is based 
on two assumptions. The first is that there is a random collection of materials in the scene. 
The second is that materials in the world are such that a random collection of them will be 
divided equally between positive and negative spectral slope. 

As suggested above, normalization requires a collection of image regions that arises 
from a random set of materials. What about using all image regions? The set of all image 
regions is not likely to represent a random collection of materials, because many materials 
will recur in several image regions. For example, if a cast shadow cuts across a single 
piece of material, that material will be twice represented, once for each side of the shadow 
edge. A second example arises with pigment density changes. In a forest scene, all 
leaves are composed of the same material (chlorophyll embedded in a cellulose base). A 
sensible normalization scheme would not take each leaf as a distinct patch of material; 
minor variations in pigment density from leaf to leaf ought to be ignored. 
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It seems clear, then, that not all image regions should participate in normalization. 
Perhaps a subset of image regions can be found that is more likely to represent a random 
collection of materials. The spectral crosspoint offers a means of finding such a random 
subset of regions. Suppose that instead of taking each image region as a distinct material, 
we took only pairs of regions that have a spectral crosspoint on the edge between them. 
We would be guaranteed that each pair of regions would correspond to distinct materials. 
The pairs of different material regions found with the crosspoint will be the subset of image 
regions that will be used for normalization. 

Our normalization scheme works like this: Recall that we expect the regions found by 
the crosspoint to represent a random collection of materials. So we expect roughly the 
same number of regions having positive spectral slope as negative. For the subset of image 
regions defined by the crosspoint, tally the number having positive spectral slope and the 
number having negative slope. If the numbers are approximately equal, our expectation has 
been met; we can infer that the illuminant is white (spectrally flat).^ Suppose to the contrary 
that the number of regions of positive spectral slope exceeds the number of negative-slope 
regions. Then we can infer that the illuminant is more intense at long wavelengths than 
at short. (Positive spectral slope means greater intensity in the longer wavelength sample.) 
Now multiplicatively scale one of the spectral samples. In the example here, we need to 
multiply all long wavelength samples by some number less than one. Exactly which number? 
The one that will fulfill our expectation of equal numbers of positive and negative spectral 
slope. That is, multiply all long wavelength samples by some number (less than one) such 
that half of the regions under consideration will have greater intensity in the modified long 
wavelength sample than the short wavelength sample, and half, the reverse. For a large 
number of samples, the multiplicative constant of normalization can be calculated from 
the mean value of the spectral slopes of all regions participating in crosspoints. See the 
algorithm for spectral normalization in Appendix 2. 

This crosspoint normalization scheme has some useful properties. Each image region 
used has the same potency in normalization, regardless of the size of the region. That is, 
each pair of image regions (found with the crosspoint) maps to a pair of data points, one 
for each region. This is good for two reasons. First, the scheme is independent of image 
region areas. This is desirable since we would not want visual systems to treat an image of 
a large blue thing and a small red thing differently from an image of a small blue thing and 

''Note there must be some crosspoints for normalization to proceed. If there are no crosspoints, 
there are no regions to consider. So although it is technically true that there are equal numbers 
of positive-slope regions and negative-slope regions (namely, zero), we do not want to infer the 
illuminant is white for two reasons. First, we have no information about any image region, and thus 
it seems imprudent to guess blindly that the light is white. Second, we have evidence that the scene 
consists of a single material since it has no crosspoints. Normalization would bring about material 
change assertions via the opposite slope sign condition, in contradiction to the evidence of uniformity 
from the crosspoint. 
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It is worth comparing our crosspoint normalization with Land's latest normalization 
theory. Land's (1983) scheme involves comparing the image intensity of a target region 
with that of a few hundred random locations in the image. In such a theory, the larger 
an image region, the more random locations it will contain. Land's theory is therefore 
area-based, while ours is independent of the particular sizes of image regions. Our theory 
makes different predictions from Land's: we expect no effect on normalization from the sizes 
of image regions, or from the lengths of image edge segments. 



5.0 Choosing a Representation 

Assume now that the image has been normalized using the spectral crosspoint 
condition, as described in section 4. We next select a representation of spectral information 
based on that rule. In particular, we seek a simple, convenient spectral representation of 
materials that is invariant under shadow, highlight, surface orientation change, and pigment 
density change. 

For any region in the image, intensity can be measured at a long wavelength and at 
a second, shorter wavelength. Call these two measurements of image intensity L and S, 
respectively, for each image region. Suppose we'd like to represent the spectral character 
of a region with a single number, namely some mapping of the pair [L, S). Furthermore, we 
would like the mapping (L, S) to be invariant under the lawful changes. The recognition of 
material differences would be easy in such a representation. A single material in its different 
guises— fully lit, shadowed, having different densities of pigmentation, with different surface 
orientations— would map ideally to a single point. If there were such a mapping, then 
whenever two image regions mapped to distinct points, we would know they corresponded 
to distinct materials. 

The lawful edge types are unfortunately so diverse that there is no function giving us the 
desired mapping. No single continuous function of {L, S) will be invariant under multiplicative 
(shadow), exponential (pigment density), and additive (highlight) changes. Material change, 
then, cannot be reduced to the problem of distinguishing two points in the range of some 
function. 

The problem isn't hopeless, however, for there is a continuous function invariant under 
some of the lawful changes, namely the multiplicative ones (shadow and surface orientation 
change). Consider again the two image intensity samples S and L. The quotient ^ will have 
the identical value on both sides of a surface orientation change or a shadow edge. The 
simple quotient is, of course, not unique in remaining constant across an orientation edge. 
Many functions of the two samples L and S have the same property. We will choose among 
three simple functions having this property: 
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Many functions of the two samples /. and S have the same property. We will choose among 
three simple functions having this property: 

S L+S L+S ^ ' 

How can we select among these candidates? The function ^ takes image regions 
into the unbounded interval (0, oo), while the other two functions take image intensities into 
closed intervals. (j~^ maps intensities into [0, 1]; ~f maps into [-1, 1].) The function J will 
be rejected, since any reasonable computational system will be better off using quantities 
that fall within a closed interval, rather than those that could be arbitrarily large. To choose 
between the two remaining candidate functions we consider the ease of discovering material 
changes in these two maps. In particular, how does the opposite slope sign condition appear 
in each of the candidate mappings? 

Given two image regions X and Y, let F denote the function -j^, so that F{X) and 
F{Y) are the values of the function F of regions X and K, respectively. Then for F, the 
opposite slope sign condition is expressed by [fli(jn{F{X)-\) -^ aign{F{Y)- W (The reason 
for this expression is that the function F takes on the value \ whenever L — S.) 

Let 6' denote the function ~~, a common measure of contrast. This is a simple 
function that facilitates the computation of material change. The sign of G is the sign of 
the spectral slope of an image region. That is, [si(jn{G{X)) y^ sign{G{Y))] emerges as the 
opposite slope (material change) condition. 

We prefer the function G to the F for our representation. Whereas to determine 
material change with G requires only a sign check, with F, the system must maintain the 
constant ^ and perform two subtractions. The particular choice of F or G, though, seems 
not to be critical for the goals we have in mind. 

Figure 3 shows the interval [-1,1], the range of the function G. Two image regions 
corresponding to lit and shadowed versions of the same material, or two different surface 
orientations, will, by design of G, be mapped to the same point. This is shown in Fig. 3a. 
Two image regions of different pigment density have the same slope sign; hence, in the G 
map, the corresponding pair of points cannot straddle the zero. The same holds for a pair 
of points corresponding to a highlight and a neighboring matte region. The latter two edge 
types are shown in the G mapping in Fig. 3b. If two image regions are mapped to points 
straddling the zero (Fig. 3c), they arise from different materials. 

To summarize, we sought a function of spectral information invariant over the lawful 
changes. That goal being impossible^ we chose {^| for two reasons. First, it is invariant 
across shadows and surface orientation changes. Second, finding material changes with 
the opposite slope sign condition is easy. The range of the function can be divided into 
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Figure 3 How various processes appear in the spectral representation implied by the mapping f^,, 
the range of which is 1-1,1 J. a) Two image regions differing only in surface orientation or sfmdow nTap 
to a single point, b) Two regions differing as matte and highlighted, or as two different degrees of 
pigmentation density, map to the same half of the range, i.e., they map to points having same-sign 
coordinates, c) Only two different materials can map to points straddling the zero, i.e., to points of 
different-sign coordinates. 



two parts, (-1,0) and (0, 1). Materials with albedoes of positive spectral slope sign will map 
into the positive half of the range, and negative-sloping albedoes to the negative part of the 
range.^ 

Finally, it's worth reiterating why we built our spectral representation around the 
opposite slope sign condition, and not the spectral crosspoint. Spectral slope sign is an 
invariant property of a material's albedo function." The opposite slope sign condition can 
be decomposed into separate meaningful statements about properties of two image regions: 
The slope sign of one region is positive, and that of the other, negative. We know something 
about each region. The crosspoint, by contrast, hopelessly confounds spatial and spectral 
information. Higher goals of color vision involve describing the properties of individual 
image regions, and cannot be reached by the crosspoint alone. 



•"Many continuous maps share the same invariance. We selected our map on the basis of algorithmic 
considerations. The particular choice is independent of the theory of finding material change edges. 

"Since a material is defined as a kind of stuff, a single material can have different albedoes as pigment 
density changes. What stays constant over these changes in density of pigment is spectral slope sign. 



10 
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Figu re 4 Steps in the construction of the trichromatic material representation, a) Two axes comparing 
L and M, and S and M samples, are joined orthogonally. Each quadrant is a material category. 
Points in different quadrants correspond to distinct materials. Points within one quadrant may belong 
to the same material; they are considered equivalent in this representation, b) Ttie line of unit slope 
in the figure above represents the comparison between S and /. samples. Adding the unit slope line 
divides the color space into six regions or "hextants." Points in different hextants arise from different 
materials. Note the hextants do not have equal areas. 

6.0 Trichromacy: Finding More Material Changes 

Suppose we add a third spectral sample, call it M, to our original S and L samples. 
Adding a third spectral sample will allow the detection of new kinds of material changes. ^° 
However, more importantly, the number of basic material categories will be increased from 
two to six. 

In the two-wavelength-sample material representation, an image region is encoded 
essentially by the rank order of the spectral samples, or equivalently by the sign of the slope 
of the line segment connecting the samples. Thus, given two wavelength measurements, 
there are two types of material— negative slope and positive slope. With three wavelength 
samples, an image region is associated with three slope signs — a slope between each pair 
of samples {SM, ML, SL). There are six possible rank orderings of the measurements 
(3! =- 6), and thus six possible basic material types. Any two regions that produce distinct 
rank orderings of the wavelength samples will bring about one or more opposite slope signs. 
Any two such regions must therefore be distinct materials. 

As a first step in constructing the trichromatic material representation, we combine 
slope information from two of the three pairs of samples. Arbitrarily, we begin with SM 



^"The additional number of material changes detected with each new spectral sample will drop 
sharply after the third sample. The reason is that the albedoes of natural objects (in the visible range) 
are typically slow-changing functions of v/avelength (Krinov, 1971; Snodderly, 1979). Cohen (1964) 
showed that three carefully chosen functions of wavelength captured over 99% of the albedo functions 
of Munsell chips. 
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and LM, combining the information in a two-dimensional space as shown in Fig. 4a. Image 
regions are mapped to points in the square [-1, 1] X [-1, 1], and a pair of points separated 
by an axis (or both axes) correspond to two regions of different material, just as did a pair of 
points straddling the zero in Fig. 3c. Any pair of points in a single quadrant may arise from a 
single material. This is the sense in which quadrants represent material categories. Without 
yet considering comparisons between 6' and L samples, we already have a categorical 
representation in Fig. 4a, in which in each quadrant corresponds to a material category. 

Let's now examine the third pairing of samples, S and L. What condition holding 
between a pair of points in the preliminary representation of Fig. 4a corresponds to the 
opposite slope sign condition between S and L? It is easily shown that if a pair of 
points straddles the line of unit slope, the points arise from materials with opposite {S and 
L) slopes.'^ Furthermore, not just the sign, but the continuous value ^ of the L to S 
comparison is contained implicitly in the representation defined by ordered pairs (|^, j^) 
that Fig. 4a illustrates.^^ 

The unit slope line in the SM-LM space therefore has special significance, and is 
added to the representation as a third material change axis in Fig. 4b. A pair of points lying 
across any of the three axes will correspond to distinct materials. Thus, each of the six 
sectors of Fig. 4b corresponds to a material type, or equivalently, to a rank ordering of the 
three samples. The particular rank ordering associated with each "hextant" is shown in Fig. 
4b. Note the hextants of Fig. 4b do not have equal areas. The original pair of axes can be 
joined in a skew fashion to allocate more or less area to the different material categories. 

To summarize, image intensities are measured at S, M and L, normalized according 
to the crosspoint normalization of section 4, and mapped to (^^, |^) in a rectangular 
coordinate system, initially creating four basic material types. A further subdivision into six 
types can arise by using the line of unit slope as a third axis, dividing the region [-1, i]^ 
into six regions, each corresponding to a different material type. Points in different hextants 
arise from different materials, whereas points common to one hextant may arise from lawful 
edge events occurring on a single material. 

Algorithm aficionados should turn to Appendix 2, where we sketch a procedure for 
spectral categorization based on the above theory. 



or 



The line of unit slope is given by §=-^ ^ ^. This is equivalent to (5- 7W)(A.+ Af) = {S+M){L-M), 
S -= L. Points above this unit slope line correspond to /. > S, points below to 5 > L. 



'' Given the values (f:,-^, j;^^-), we can compute the value of ^. Let Q = |^ and R = f^ 



12 



RUBIN AND RICHARDS COLOR AND MATERIAL CATEGORIES 

7.0 Relation to Psychophysics and Neurophysiology 

Our spectral representation of material types is but an abstract model of biological 
color vision. In our theory, certain terms are left undefined. We haven't described what the 
"spectral samples" of the theory are, and v^e haven't said anything about how materials are 
encoded. How then can we assess its relevance? Two linking assumptions will guide the 
interpretation of our theory. First, in the discussion of the psychology of color vision, we 
will argue that of the traditional color variables hue, saturation, and lightness, it is hue that 
encodes material type. Second, in the discussion of neurophysiology, we take the small step 
to identify the spectral samples of our theory with the relative stimulation of the three human 
cone photopigments (or combinations thereof). ^^ Given this interpretation of our theory, it 
turns out that double-opponent units found in color neurophysiology can be understood as 
performing the spectral crosspoint and/or the opposite slope sign computation. 



7. 1 Psychologically Unique Primaries 

Ewald Hering (1964) offered a psychological account of human color perception that 
was based on the notion of opponent processes. He observed that "redness and greenness, 
or yellowness and blueness are never simultaneously evident in any color, but rather appear 
to be mutually exclusive." This is a clear case of categorical perception. Reddish and 
greenish are mutually exclusive hue categories, and if hue is encoding material properties, 
then the two categories will partition materials. See Fig. 5a. Similarly, bluish and yellowish 
will partition materials. See Fig. 5b. These two sets of mutually exclusive hue pairs divide 
the color space into four regions, as in Fig. 5c, just as did our trichromatic color space (Fig. 
4a). 

Our claim that Hering's color quadrants correspond to our material categories is 
predictive: we expect that shadows, surface orientation changes, and pigment density 
changes would only rarely cause perceived hue to change from reddish to greenish (or vice 
versa), or from yellowish to bluish (or vice versa). As noted in Appendix I, highlights could 
be troublesome. 

The fact that there are four hue categories supports the idea that trichromatic human 

vision uses two opposite slope sign checks, as in Fig. 4a, but not the third, as shown in 

Fig. 4b. (Goethe [1808], however, proposed a theory of color perception based on six 

hue categories, which might correspond to the use of all three opposite slope sign checks.) 

^^Our theory of crosspoints and opposite slope signs was based on spectral samples at a single 
wavelength. Biological measurements of the spectrum are broadband. It turns out that broadband 
samples cannot introduce crosspoints that are false targets. That is, a spectral crosspoint found with 
broadband samples is still a reliable indicator of material change (Rubin & Richards, 1982, Appendix 
IV). The opposite slope sign condition may not be as robust; more work is needed to study the effects 
of broadband sampling. 
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Figure 5 Hering's notion of opponent color processes, a) All colors are either reddish or greenish, 
but never both, b) All colors are either bluish or yellowish, but never both, c) The two pairs of mutually 
exclusive colors divide the color circle into four quadrants, similar to the trichromatic representation 
that we develop in Fig. 6a. 



Evidence from infants (Bernstein et al., 1976) supports Hering's theory of four hue categories 
as independent of language and culture. Pigeons also have categorical color perception 
(Wright & Gumming, 1971), suggesting the computational scheme that we propose here is 
fundamental to color vision across species. 

Hering's notion of opponent color processes implies four special hues. They are 
indicated in Fig. 5c. These hues, which Hering called psychological primaries, are the 
boundaries that separate color categories. Primary red is that hue among the reddish hues 
that separates the yellowish from the bluish; primary blue is that hue among the bluish 
that splits the reddish from the greenish; and so on. These primary colors are unstable 
in the sense that any deviation from them involves a change of color categories. Hering's 
psychological primaries correspond to the axes of our trichromatic representation (Fig. 4a). 

Just why these primaries have their particular locations in the spectrum is an interesting 
evolutionary question not addressed here. One possibility is that a creature's material 
boundaries are positioned in some way as to make the greatest number of discriminations 
among materials encountered in its environment.^'' Interesting work has been done along 
these lines. Snodderly (1979) attempted to relate the color vision of New World monkeys to 
the spectral characteristics of their jungle habitat. Levine & MacNichol (1982) and McFarland 



'•^Material boundaries can be changed in two ways. The wavelength at which a photopigment captures 
the greatest percentage of photons can be altered, or new "channels" can be created by combining 
photopigments. One sort of combination of two spectral samples S and /> is a rotation; that is, new 
coordinates {S cos - Lsin 0, S sin + Lcos 0) can be created for some angle of rotation 0. The original 
and rotated coordinate systems will not always agree about whether two image regions satisfy the 
opposite slope sign condition. That is, the two spectral coordinate systems differing only by a rotation 
will make different material distinctions. An angle can therefore be selected to maximize the number 
of material changes detected. 
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& Munz (1975b) linked the photopigment cliaracteristics of fishes to the spectral character 
of light in their environments. 

In sum, our spectral representation of material categories is a two-dimensional space 
in which each quadrant represents a material type, and the axes represent the boundaries 
between categories. Image regions that map to different quadrants necessarily arise from 
distinct materials; image regions that map to the same quadrant may arise from a single 
material. Supposing that hue encodes material information, Hering's observation about 
human color vision makes sense: hues are divided into four fundamental categories by the 
mutually exclusive pairs red-green and blue-yel!ow. 

7.2 Land's Experiments 

7.2.1 Two-Color Projection 

Edwin Land (1959a,b) conducted some remarkable experiments in two-color projection 
of natural images. Some of the phenomena he reported can be understood in terms of our 
"materialistic" theory of categorical color vision. 

Land's paradigm was as follows. Two different black-and-white transparencies were 
made of a colorful natural scene by means of long- and short-v/avelength filters.'^ The two 
transparencies were called the long and short records, respectively. Corresponding regions 
of the two records, in general, were of different grey values. The two records v/ere projected 
on a screen in register, the short record with short wavelength light, the long record with 
long v^avelength light. Surprisingly, the resulting image was richly colored and faithful to 
the original still-life. 

Land's (1959a,b) work was basically descriptive. He found a means of predicting the 
hue name of a region in the two-color reconstruction. The intensity of long-wavelength 
light in the region was expressed as a fraction of the maximum long-wavelength intensity 
in the entire image. The same was done for short-wavelength intensity, yielding a pair of 
numbers (each between and 1). This pair of numbers (fraction of maximum S, fraction of 
maximum L), plotted on log-log axes, yielded a coordinate system that Land used to relate 
image intensity to perceived hue. Land's coordinate system (hereafter called "Landspace") 
is shown in Fig. 6a. 

We will now try to relate our current work to Land's findings. Whereas Land began with 
some surprising experimental observations of color appearance, we took image intensity 



'^The transparencies did not consist solely of black regions and white regions, but rather the full 
range of grey values between black and white. 
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equations as the starting point of our theoretical investigation of the problem of discriminating 
materials. We will show how these two approaches dovetail. 

Our argument below consists of four major points. First, we look at how the spectral 
crosspoint appears in Landspace. Second, we propose that an absence of crosspoints 
should cause a total failure of Landspace, and note the failure conditions already observed 
for Landspace correspond to such an absence. Accordingly, we make some predictions 
for two-color projection that conflict with predictions in the literature. Third, we note 
the opposite slope sign condition is identical to the fundamental split between warm and 
cool color categories in Landspace. Finally, we suggest a straightforward extension of 
our crosspoint normalization theory that would account for a peculiar result in two-color 
projection. 



7.2.2 The Spectral Crosspoint in Landspace 

In general, the light source for a scene will not be white. (A white source is one that 
emits the same flux of photons at each wavelength.) Suppose we take two spectral samples 
of image intensity S and L. Spectral normalization is any procedure that transforms S and 
L into new values S* and L*, where the latter measurements would have been obtained had 
the illuminant been white. 

Land's normalization is {S*,L*) = (^, j^), where Smax and L^ax are the greatest 
intensities measured in the S and L samples throughout the image. 

Our theory of normalization is based on the spectral crosspoint, as discussed in section 
4. To relate our current work to Land's experiments, we must ask how spectral crosspoints 
appear in Landspace. We claim that a crosspoint corresponds to a pair of points in 
Landspace that form a line segment of negative slope (in Landspace). To avoid confusion, 
we will refer to the slope of line segments in Landspace as "Landslope," as distinguished 
from spectral slope in plots of intensity versus wavelength as discussed earlier in the paper. 
(Landslope, then, is a function of a pair of regions, whereas spectral slope is a property of 
a single region.) Our claim, again, is that a spectral crosspoint corresponds in Landspace 
to a pair of points of negative Landslope. The proof follows. 

Suppose there is a crosspoint between regions X and Y. Then, say, Sx > Sy and 
Lx < Ly. Does the crosspoint imply some sort of relationship among the Landspace 
coordinates for X and Y, {S*^,L*j^) and (5"^.,/.;^)? It's easy to see from the definition of 
Landspace coordinates that S*x > Sy and L*x < Ly- Now Landslope is given by 4'f^=4f . 
so the Landslope of crosspoint regions X and Y is negative. (Note the assignment of S* to 
the abscissa is irrelevant to the result.) 
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Figure 6 Landspace and some of its achromatic loci, as discovered by Land (1959a). A) Land's 
coordinate system (adapted from Fig. 1 of Land, 1959b) that relates perceived hue to the fraction of 
maximum long- and short-wavelength light (expressed on log-log axes). This coordinate system we 
call "Landspace." B) Image regions correspond to a line of unit Landslope. Such an image (as well 
as the next two) results in a monochromatic percept. (This is produced by placing identical records 
in the long- and short-wavelength projectors.) C) A line of zero slope. (This is created by removing 
the record from the long-wavelength projector.) D) A line of slope -l. (One record is placed in the 
short-wavelength projector, and its photographic negative is placed in the long-wavelength projector.) 



7.2.3 Failures of Landspace 

Landspace is a way of predicting the perceived hue of a region given the ratio of its 
Intensity to the maximum intensity, at long and short wavelengths. This predictive scheme 
is successful for two-color projection of natural images. Land noticed, however, that for 
certain contrived images, his coordinate system failed totally. These images were seen 
as achromatic (or monochromatic). What did these concocted failure conditions have in 
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common? Land (1959a) suspected that "any arrangement which yielded points falling on 
a straight line [in Landspace], or even on a simple smooth curve, would be colorless." 
[Judd (1960) formalized Land's results on failure conditions.] We will show that the failure 
conditions Land has discovered correspond to situations in which our theory is unable to 
make any material distinctions. Furthermore, we will show our theory predicts stricter failure 
conditions than does Land in his conjecture.'* 

Figures 8b,c,d depict three concocted situations that Land found [and Judd (1960) 
verified] to cause a breakdown of Landspace. In figures 6b and 6c, the failure loci are 
straight lines of non-negative Landslope. Notice that for such loci, there can be no spectral 
crosspoints, since crosspoints correspond to point-pairs of negative Landslope. 

Our normalization scheme, re-cast in Landspace, calls for inspection of all point-pairs 
of negative Landslope, since this subset of points is more likely to arise from a random set of 
materials than the totality of points. So a visual system using our normalization procedure, 
finding no point-pairs of negative Landslope (no crosspoints), would fairly conclude that 
there are no material changes and hence only a single material is present. A monochromatic 
(or achromatic) percept is an apt result, then, for a system that encodes material type by 
hue. 

Consider next a collinear collection of points of negative Landslope in Landspace. 
Normalization can proceed according to our scheme, since spectral crosspoints are avail- 
able. Thus we disagree with Land's (1959a) conjecture that all collinear sets of points will 
be failures. We predict the locus shown in Fig. 7a will yield a range of hues. Only collinear 
sets of positive Landslope will fail. 

There is one special exception to our prediction that collinear loci of negative Landslope 
will produce chromatic percepts. A set of points of Landslope -1 (Fig. 6d) corresponds to 
an isoluminance image. Such an image has no luminance edges, and has long been known 
to disrupt vision (Evans, 1948). We have argued elsewhere (Rubin & Richards, 1982) that 
crosspoints are only meaningful across edges, and hence should only be sought across 
luminance discontinuities. Thus the isoluminance condition (the locus of Landslope -1) 
implies an absence of crosspoints and a failure of normalization, leading to an achromatic 
percept. 

We turn next to Land's conjecture that curved loci in Landspace will yield achromatic 

percepts.'^ We believe this is an overgeneralization. We predict, along with Land, that the 

'^'Land's conjecture (that smooth one-dimensional loci in Landspace will be seen as achromatic) 
is problematic. It seems difficult to legisiat0 whether a collection of points in a plane constitutes a 
curvilinear arrangement or defines an area. A smooth curve can be drawn through any collection of 
points in a plane. 

'■^A line in Landspace does not have absolute significance anyway, since linehood depends on the 
choice of axes. For example, a line in Landspace with log-log axes will not be a line with linear or 
power axes. 
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Figure 7 Predictions of our theory that conflict with Land's conjecture that all one-dimensional loci 
in Landspace will yield achromatic (monochromatic) percepts. A) A linear locus of negative Landslope 
(t^ - 1). B) A smooth locus of points without point-pairs of negative Landslope should t>e a failure 
condition. C) A smooth locus of points that has point-pairs of negative Landslope should produce a 
range of hues. 

non-linear locus of points in Landspace shown In Fig. 7b, since It contains no point-pairs of 
negative Landslope (no crosspoints), Will be achromatic. In contrast, the one-dlmenslonal 
locus of Fig. 7c has point-pairs of negative Landslope, and should yield a range of hues. 

To sum up, we have suggested that failure conditions of two-color projection occur 
when there are no spectral crosspoints* That is. Land failures should occur if there are no 
(or too few) point-pairs of negative Linjdslope. Our predicted range of failure conditions Is 
therefore narrower than Land's. Furth^rlrjore, Land's account of failures is purely descriptive; 
ours is explanatory (via the theory of jnMerial changes). 



7.2.4 Opposite Slope Sign and Landspace 

We have argued that the opposite ^lope sign condition (between two regions) is strong 
grounds for inferring the two regions air^ composed of different materials. Can this condition 
be recast in Landspace? The answer ts| yes: two regions in the opposite slope sign relation 
map to two points straddling the line i)fl, unit Landslope in Landspace.^^ 

The argument Is as follows. Inn^ye regions X and Y satisfy the opposite slope sign 



^*Land's early work relied on two spectr4l samples. Thus there is only one opposite slope sign 
condition to worry about, as shown in F|git 5. Our trichromatic theory, sketched in Fig. 6, is not 
applicable to Land's work. 
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many complex natural images, Land's 



condition if normalized image intensities'*' obey the following: L*x > -^*x ^"ci L*y < Sy, 
where L*x denotes the normalized inteisity in the longwave sample of region X, and so 
on. But the last condition indicates that {S*x,L*x) lies above the line of unit Landslope 
in Landspace (given the abscissa marl^s S* values), and {S*y,L*y) lies below. If the Land 
normalization is correct, then we ha\^ shown that two regions in an oppo$ite slope sign 
condition map to a pair of points in Laidspace straddling the line of unit L^indslope. (For 



normalization scheme and ours coujd yield similar 



results. That is why we can accept La.nJ's scheme as approximately correct!) 

Examine again Land's results shcjwn in Fig. 6a. Land observed that the l^ues appearing 
above the line of unit Landslope are al|l Twarm," and those falling below are 'Icool." (Wilson 
& Brocklebank [1960], in a study of tWo-color projection phenomena, notec) that although 
hue, saturation, and lightness were n()t precisely preserved in the two-color reconstruction 
of the original still-life, at least the warn/cool aspect of hue was invariant.) ifhe distinction 
between warm and cool colors is cej^tainly the most fundamental fact of categorical hue 
perception. To sum up, given that Lund's normalization has been succdssful, different 
materials (as discovered by the opp(l)site slope sign criterion) map in Landspace to two 
points straddling the line of unit Landslope (and vice versa). In turn, two points straddling 
the unit slope line correspond to two (|ualitatively distinct hues, one warm and one cool. 
This observation supports our claim that hue is encoding information abou^ differences in 
material. 
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7.3 Neurojonysiological Operators 
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non-zero product implies a crosspoint. 



23 



AL CATEGORIES 



22 



nits described 

That is, both 

one spectral 

spatial field 

sufficiently 

crosspoints, 



to or 1 
—the modified 
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8.0 Summary 



sijn 



Our theory of color vision present; ; 
normalization and the opposite slope 
illuminant and to categorize the albedbes 
between the common natural pigments 
example, but not between the density 
does not address this latter problem- 
grain of a piece of teakwood. A quantitative 
the qualitative computations described 
Categorical color vision is simply an 
coarse judgments about materials. 



two types of operators— the spectra 
■which suffice in most cases to 
in the scene. Our scheme shoijld 
(chlorophylls, xanthophylls and 
\[ariations of any one of these pigmerjts 
lamely how we appreciate the fine 
color vision system, of greater 
here, will be needed for such fine 
nexpensive method for making 
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crosspoint for 

n(|)rmalize for the 

differentiate 

f|lavanoids), for 

The theory 

(bhanges in the 

complexity than 

qiscriminations. 

and reliable 



rapid 
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Appendix 



ecges 



This appendix shows that Image 
2) pigment density variations, 3) shadows 
of image intensities across that edge, 
condition. 



1.0 Surface Orientation Change 



Let X and Y be regions on either 
discontinuity. Then the image intensifies 
measured in X and Y, respectively, are 
some constant a (Rubin & Richards, 1 
only by a multiplicative constant have 



rface 



side of an edge due solely to a su 

(as functions of v/avelength) Ix 
related multiplicatively. That is, /x() ) 
; Horn & Sjoberg, 1979). Two funiti 
identical ordinality. 



882; 



2.0 Pigment Density Variation 

Suppose X and Y are two region; j 
only in pigment density. Then if the albeqo 
the albedo of Y can be approximated^^ 
density (Rubin & Richards, 1982; Wysze 



The light measured from regions X 
with the radiant intensity of the illuminan 
pigment density change is stipulated as 
the same for both, then any difference 
will be due to a difference in albedo 
an exponential constant, and two functions 
Image intensities across a pigment dens 
of this relation for natural pigments can 
(1975) or Snodderly (1979).] 



3.0 Shadow 

Consider an edge separating a lit 
regions reflect diffuse illumination toward 



'^^This exponential relation presumes that 
embedding layer reflects different wavelengtht 



the embedding material is spectrally 
unequally, then change in pigment density 
complex descnption. In particular, pigment density changes can mimic material changes 



25 



COLOR AND MATER AL CATEGORIES 



: Lawful Processes 



that arise from 1) change in su 
and 4) highlights all preserve the 
and hence cannot cause the 



rface 



orientation, 

o|rdinal relations 

opposite slope sign 



on a planar piece of a single material that differ 

(as a function of wavelength) of region X is p{\), 

by /(X), where 6 is a constant related to pigment 

:ki & Stiles, 1967). 



the 



and Y is the product of the albedoes 
. Since X and Y are assumed coplaqiar 
the sole cause of the edge), and the 
between measured intensities from 
ftlnctions. But the albedo functions 
so related have Identical ordlnali 
ty change will have identical ordinali|ty 
be seen in Krinov (1971), Francis 



111 



region from a shaded one. Both 
the viewer. The lit region, in additibn 



orientation 

(X) and /y(X), 

= a/r(X) for 

ons differing 



of X and Y 

(recall that 

illumination Is 

two regions 

Are related by 

ty. Therefore, 

[Examples 

& Clydesdale 



and shaded 
, reflects a 



nieutral. If the 
has a more 
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direct source. If f/.<(X) and fshad{\) are i|nage intensities (as functions of wavelength X) from 
lit and shaded regions, respectively, then: 



hhadi^) 
Jlit{\) 



ai 



E 



where Ediffu8e{^) and /id,r,rf(X) are th^ 
p{\) characterizes the albedo of the m^^erial 



By inspection of equations (1), it 
of shadow. That is, a false target is 
regularities. There is usually some ctp 
(Goral et al., 1984). This is not surpri^i 
reflections of the direct light from a 
be made that this is usually the case: 
the same spectral character as the di 
constant k. This we call the "grey 
implied by the statement that all the 
data support the grey world assumpti 
functions in a pine woods in a sunny 
similar in shape, and are shown in Fig 



re;t 



en. 



area 



Invoking the grey world assumption 

hhadi^) 4= 

//.t(X) 



Note that the lit and shaded 
intensity functions. Ordinality will therefclre 



4.0 Highlights 

The analysis of highlights is slightly 
Richards, 1982; equations 14a) express the 
neighboring matte region: 



frnattei^) — {^^diffuaeO<] 
highUghti^) — ^^directi^) 



where Irnatte{>^) and Ihighiight{>^) are the 
matte and highlighted regions, and 6 e{(l 
surface is mirrorlike {6 = 1 describes a 
1982, for a more extended treatment.) 
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//u«e(X)/?(X) 

:t7/u8e(X) + ii'd,rec<(X)]p(X) 



(1) 

diffuse and direct components of illumination, and 



irect 



assu 



s clear that ordinality can be violated 
possible. The visual world, fortunately, 
1 36 relation between diffuse and di 
ng, since diffuse light results from di 
v$i|iety of materials in the scene. An 

visual system can presume that diffuse 
light. That is, Eai/fusei^) = f^I'^dire 
wd)rld" assumption (see Section 3.1), 
alqedoes of a scene will average to 

Mailman (1979) measured speciral 
and in nearby shade. The functions 



gnjy 



in the case 

offers certain 

illumination 

/erse, random 

mption will 

light has 

t(X), for some 

because it is 

Anecdotal 

irradiance 

are strikingly 



, equations (1) become: 

kEdirect{'>^)p{\) 

(1 + lc)Edirect{\)p{\) 



regiops now give rise to multiplicatively ("elated image 
be preserved. 



more complex. The following equations (Rubin & 
image intensities to be found in a highlight and 



+ ^dtVec<(X))/9(X) 

- (1 - S)[Eaiffuse{\) + ^rf.rect(X)]/.(X) 



mages intensities (as functions of 
1) is a constant that indicates to 
)erfect mirror). (See Richards, Rubih 
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(3) 



w|avelength) in 

extent the 

& Hoffman, 
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Shadov/ and Wavelength 



o 
"o 

o 




Figure 9 Measurements of the spectral ii*^ridiance 
Florida pine woods, adapted from Hailman {f979) 
flux. The abscissa shows wavelength. 



•^.., Lit 



Shadow 



\. 



V 



Wavelength 



functions of direct sunlight and n 
, Fig. 74a. On the ordinate is the 



logi iri 



I >arby shade in a 
ithm of photon 



The equations express the fact that both highlighted and matte regions reflect 
and diffuse light. In addition, the highlight, acting as a partial mirror, reflects 



tie 



Applying the grey world assumptio i, equations (3) become 

/matte(X) = (l + A;] Edt>ect(X)/j(X) 
highlightO^) = SEdireli^) + (1 " S){l + k)Eairect{\)p{\) 



which reduces to 



^mafte(X) = (1 
highUghti^) = E^ 



■H k)Edirect{\)p{\) 
ect{\)[e + (1 - 8){\ + k)p{\)] 



u; 



By inspecting equations (5), it car 
violation in ordinality. Assume now that 
color of the illumlnant. Normalization 
character of the illuminant. (Such a corjiputat 
is equivalent to a transformation of the 
the illuminant been white; it allows us to 



both direct 
direct light. 



(4) 



(5) 



be seen that highlights can prod 

he image has been normalized with 

any scheme that allows recovery 

ion is presented in section 4.) 

ii|nage Intensities to what they would 

set Edirect{><) = /?, where /3 is some 



u:e 



a spurious 
respect to the 

df the spectral 
Normalization 

Have been had 

constant. 



Both equations (5) can novy be rew-ltten substituting /3 for Edirect{>^), yielding 
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hiighlighti^) 



With the two assumptions of grey 
produce violations in ordinality. This caili 
function of the highlighted region is sinply 
neighboring matte region. The intensity 
(1 - 6), and then a constant function (/ 
ordinality; hence no opposite slopes wil 



^orld and spectral normalization, hig|hlights 
be seen in equations (6), where the 
related to the image intensity 
in the matte region is multiplied 
X) = sp) is added. These two operations 
arise given our assumptions. 
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/?(l+A;V(X) 

f3[S + {\ - 5){l + k)p{\)] 



(6) 

will not 

image intensity 

i unction of the 

by a constant 

preserve 
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Algorithm for 
and Mate 



Regions in different categories are made 
materials is sketched below. The first gt 
is to categorize. 



Given a full-color image of a scenci lit by an unknown illuminant, and a 
edges and regions, regions can be ass gned to one of a small number of mateHal 
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I^Dpendix 2: 
Spectral Normalization 
al Categorization 



of different materials. An algorithm 
3P is to correct for colored illumi 



way of finding 

categories. 

f6r categorizing 

nati(t)n; the second 



Irj 1he Beginning 

The original full-color image can be viewed through three spectral filters) yielding three 
distinct maps of image intensity, say r\ <p, and B. See Fig. 10a. These three maps of image 

c haracteristics, 



intensity we call "spectral images." The 
should not be important. All that matters 



Speckral Normalization 



First, apply an edge operator to the 
crucial. Assume the edge operator prod 



T-vertex terminates the edge that is the 
edge segments is important because we 



number of filters, or their spectral 

is that the filters yield independent rfieasurements. 



image. The particular edge operator 
jces a closed set of edges.^* Next, 



should not be 

3 segments 

must be made explicit. See Fig. 10b. |f]is involves understanding vertices. For example, a 

crossqar. Identifying 



BQ, but not the edge that's the 
will iterate through a list of them. 



side. Call 
be free of 



For each edge segment, two narrcw strips must be defined, one on e^ch 
the strips X and Y. (Understanding veiUces is important because the strips rr^ust 
edges.) See Fig. 10c. 

Average the intensity values of each of the spectral images R, C, and B in both the X 
and Y strips. The output of this step i^ i ix values Rx, Ry, Gx,Gy, Bx, and By. 



r two types of crosspoint, RG, an|j 
and [Gx - Gy)[Bx - By) < 0, respect 
ing the R and B samples. 



For each edge segment, check |fc 
conditions are {Rx - fW){Gx - Gy) < jO 
the possibility of a third crosspoint invoh 

Suppose an image has n crossoo 

^''If algorithm for edge detection does no( j)roduce closed edges, then regions 
identified using edge fragments. 

^•"'The R and C samples can yield crosspoiKt 



G' sample could just as easily be taken as the photopic luminosity function 



nt edge segments. For each crosspoint, record 

mu^t somehow be 



>, and independently, so can the B and C samples. The 
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(A) 





B 



i 



i 



Full -color image 



(B) 




Spectral images 



Edges, vertices 
are made explicit 



(C) 



i- 



(D) 



R-G 
R+6 






FIgu re 1 a) The full-color image is run th 
have been found and made explicit. This Im 
and are here marked with large black dots, c; 
Y are defined. No edge segments should b0 
spectral images in both of the sirips, yioldlrjg 
in the image, d) Measurements taken from s 
defined by axes as labeled. Normalization 
equal numbers of po ints will be found in e 



a&i 



Edge strips are 
defined; each strip 
yields a point in th^ 
spectral space below 



B-6 



B+G 



Uneven distribution 
of points in a spectral 
space 



r( ugh 



three spectral filters R, G, and B. b) Edge segments 

shows five edge segments. Vertices hgve been found, 

On either side of one of the edges, narrdw strips X and 

n the stnps. Intensity averages will be taken in the three 

six measurements. This is done for each edge segment 

about each edge map to points in a spectral space 

do(isists of multiplying li and U values by factors such that 

quadran t. 



a le 



s rips 
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spectral information about the two abu||tting strips. In particular, store two color contrast 
values per region: 



Hi — G i I ff — Gi 



Ri + Gi' Ik + Gi' 



where i is an index ranging over the i 
particular form of ratio is useful because 
spectral information recorded can be 
space (with axes of j^ and jj^) sho 

Let U be the number of points in 
and L be the number of points in the I 
a random assortment of materials to 
equally distributed among the quadrants 



n edge strips defined around n crosspoints. This 
its value must lie in the closed interval [-1, 1]. The 
cohsidered as 2n points in a two-dimensional spectral 
vn in Fig. 10d. (See also Fig. 4a.) 

tl-e 



upper half-plane of the spectral space (Fig. lOd), 
i half-plane. Under a white illuminant, we'd expect 
yjeld U ^ L ^ n] that is, points should be roughly 
of the spectral space. 



If the 2n points are not divided equjally 
must seek normalization constants a 



an J 



MEDIAN 



aRi - Gi 



aRi + Gi 



-it=i,..., 
For a large enough number of i 

1 



i+n 

where Crg and Cbg are means of the 



2n 



<^«« == ^ L. ^ 
1=1 



Ri- 



4 



The values of a and /? in (10) will 
criterion (9) will hold) given some simpli 



The correctness of the normaliz^t 
verifying that criterion (9) holds. If not, 
in an iterative procedure. The entire ncir 
11. 

Once correct values of the normali 
three spectral images R, G and B can 



^^There must be at least 12 independent 
of measurements { -J[^^^^ } must approach 
measurements { ^^^ }■ (See Siegel, 1956i) 
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i = 1, . . ., 2n 



(8) 



among the quadrants of the spectral space, we 
(3 that satisfy the following criterion: 



MEDIAN 



(3Bi - Gi 



PBi + Gi 
ma$e regions, we can take 







(9) 



t=l,...,2n 



RG 
RG 



P 



l-V 



BG 



1-\-Vbg 
iets of measurements (8): 



(10) 



Gi^ 
Gi 



p J_ v-^ Bj — Gi 

^^ ~ 2" ,^ i^i + Gi 



(11) 



provide a correct normalization (i.e., normalization 
statistical conditions.^® 



on constants a and /3 can easily be checked by 

t ie values of a and ft can be adjusted Incrementally 

nalization algorithm is shown as a flowchart in Fig. 



ation constants are returned by the algorithm, the 

be transformed into a set of normalized spectral 

( rosspoint edges, and the mean and median of the set 
he same value as i ^ 00, and similarly for the set of 
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NORMALIZATION ALGORITHM 



Begin Mith Points 
in Spectral Space 



1 -1/2n 2 



Set a: 



Set p = 



2n 



i = 1 






+ 1/2n y ^^-^ 



1=1 



2n B,-6i 



|TV»| 



J 



£ 



Complete ^ and U 

J I 



Compare 
Uand n 



Stop: 
Return a 



u<n 



Increment a 



2ir>n 

► 



Decrement a 



Compute 
qR-G 
aR*G 
ond recompute 
U 



Figure 11 Normalization Flowchart. Bedin 
a pair of multiplicative normalization coefficients 
to balance th e li image with respect to G ._ 



Stop= 
Return /3 



1 



Compere 
X and n 



Increment j3 



X'>x\ 



Decrement j3 



^<n 




with points scattered in spectral space, and end with 
a to balance the II image with respect to G, and ^ 
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FIgu re 1 2 a) The three spectral images R 
produced by the procediire shown in Fig. 1 
D' . b) The regions of the image sketched i 
assigned one of four possible ordinal doublets 



G, and D are normalized using the multiplicative constants 
1. The normalized spectral intensity maps are R* , G\ and 
1 Fig. 10b labeled with material categories. Each region is 



r> 



images. All values in the R image are 
denotes normalized intensity; see sectilon 
unchanged: G = G*. See Fig. 12a. 



ultiplied by a, yielding R*. (The asterisk superscript 
7.2.2.) Similarly, D* =- /3B. Spectral image G is 



Spectral Categories 



Suppose that when closed edge 
explicit. For each region i, measure 
yielding the triplet (<,G*,i?*). A 
relations: 



sfegments were found that image regions were made 

average values of the normalized spectral images, 

of numbers yields one obvious pair of ordinal 



th(j 



triplet 



(fi!* , G • , 73 • ) h^ [aiguRG, signBG)i 



where aiguRG is "+" if C* > R*, and " 



gned 



Each region can therefore be assic 
(-,+), (+,-)• This is shown in Fig. 12|d 
composed of distinct materials. 



Note that a third ordinal relation 
If this relation is included, six spectral 



i| sometimes independent, the R* - B* comparison, 
c ategories obtain. 



ritli 



Finally, note that while the algo 
mation has not been Ipst; it is still avai 
the continuous-valued coordinates 



(12) 



" Otherwise. 

to one of four material categories: (+, +), (-, +), 
Two regions that are in different categories are 



m described here is categorical, continuous infor- 
I able for more refined purposes. For each region i, 
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should be useful. 



Gi Bi-Gi 



+ Gi' Bi + Gi 
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(13) 
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